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Abstract 

The Dynkin isomorphism associates a Gaussian field to a Markov 
chain. These Gaussian fields can be used as priors for prediction and 
time series analysis. Dynkin's construction gives rise to Gaussian fields 
with all non-negative covariances. We extend Dynkin's construction 
(by introducing a sign structure on the Markov chain) to allow general 
covariance sign patterns. 

1 Introduction 

Let X = {X t }t>o be a reversible Markov process with a countable state space 
X and symmetric generator matrix Q = ((q X y))x,yex, such that all states are 
transient. To ensure transience of Q, it is sufficient to assume that either 

Q is irreducible and q xy < for atleast one x G X. (1) 

or 

q xy < for every x E X. (2) 

We provide a proof of this in Section 2. Under both of these assumptions, Q 
is non-conservative, i.e. from atleast one state there is a positive probability 
of going to an absorbing "cemetery" A (not included in X) and staying 
there forever. Dynkin [1] associated a Gaussian field {Z x } x£X with variance- 
covariance matrix 

S = -Q- 1 
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with this Markov process and derived various intersesting properties of this 
correspondence. Since then, this correspondence has been used in several 
contexts (See Section 3). All individual covariances of the Gaussian field 
{Z x } x( zx are non- negative in this construction. In this paper, we will extend 
Dynkin's construction to a larger class of variance-covariance matrices, which 
allow for positive as well as negative covariances. 

For this purpose, we introduce a "sign-matrix" S such that 

S(x, x) = l, S(x, y) = S(y, x), S(x, y) e {-1, 1} Vx ^ y e X. 

If S(x,y) = 1, the transition from x to y is called a positive transition. If 
S(x,y) = —1, the transition from x to y is called a negative transition. Let 
Si denote the random time corresponding to the i th jump for the Markov 

process {X t } t >o, i = 1,2,3, Define the "sign-process" H = {H t } t > 

by 

oo 

H t = n^P^.!, (with So = and H = 1). 

i=i 

To describe in words, H t = 1 if the number of negative transitions of X upto 
time t is even and H t — — 1 if the number of negative transitions of X upto 
time t is odd. Also, the transition to the "cemetery" A from any state is a 
positive transition by default. 

Consider a Gaussian field {Z^} xG x with variance-covariance matrix 

£ 5 = (-QoS)- 1 . (3) 

(We explain what we mean by (-Q o S)^ 1 when X is countably infinite in 
Section 2). As usual, let 



It [ ^{x s =x}ds, t > 0,x G X 
Jo 



denote the occupation time of the Markov process {X t }t>o in the state x till 
time t. We prove that for a realization of X independent of {Z^} xe x and for 
each bounded Borel measurable function F : — > R and x, y E X, 



E 



z s x z s y F (^iSl!, ueX^j = J E F + l u OQ ,ue X^j H, 



djJj X y, 

(4) 
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where fi xy is the conditional probability measure given that the process 
{X t } t > enters the "cemetery" A eventually with y being the last state it 
stays in before being killed, scaled by a factor of — We also prove 
identities for conditional prediction of the Gaussian field {Zf } x( zx in terms 
of the Markov process X. If A C X is finite, then 

E[Zf | Z s a , aeA} = J2 V b [l { x RA =a}H RA }ZS, VbeX\A, (5) 

where Ra is the first time (greater than or equal to Si) when the Markov 
process X hits A, and 



Cov{Zi,Z§\Z*, aeA)=E b 



oo 

l{x s =b>}H s l{ s<RA } 



V6, b' e X \ A. 

(6) 

Hence, the formulas for Gaussian field predictions in this case can be ex- 
pressed elegantly in terms of quantities related to the Markov process X. 

Note that, Dynkin's construction is a special case of (J3]) with S(x,y) = 
1, Vx ^ y 6 X. In this case H t = 1 \/t > 0. Also, the version of (jl]) when 
S(x, y) = 1 Vx 7^ y G X is known as the Dynkin's isomorphism theorem. 
It is remarkable that all changes that arise in the formulas as a result of 
introducing a "sign-matrix" are reflected by just the "sign-process" H. Note 
that it is easy to keep track of {H t }t>o while simulating {X t }t>o- 



2 Preliminaries 

We clarify what we mean by inverse of an infinite matrix, atleast the ones 
that we are dealing with. Let Q be an infinite matrix which can be written 
as R(I — P), where R is a diagonal matrix with negative entries and \P\ := 
((\Pij\))o<i,j<oo is a sub-Markov matrix satisfying XmLnJ P\ n < 00 0- e - eacn 
entry of the matrix is finite). Then the matrix (5^^°=o P n )R~ l satisfies 

qI^TpAr^ (jrP n )R~ 1 Q = I. 

\n=0 / \n=0 / 

Hence, in such cases we define 

\n=0 J 
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As in the introduction, let {X t } t > be a reversible Markov process with a 
countable state space X with symmetric generator matrix Q = ((q X y))x,yex 
satisfying ([I]) or Note that q xy = q yx > for x ^ y, q xx < and 
J2 y ex q^y — 0- Let {l^}i>o := {Xs t }i>o be the embedded discrete-time 
Markov chain with one step transition probabilities 

Pxy = ~— for x ^y, p xx = 0. 

qxx 

Let P := {{p X y))x,yex and let Q diag denote the diagonal matrix with diagonal 
entries same as Q. Then 

Q = Q dia 9(I-P). (7) 

We describe a typical path of {X t } t > . The process starts at an initial state 

Yq. The process stays at Y n during [S n ,S n+ i) for n = 0,1,2, and 

at a random time £ = S v+ i jumps from the state Y v to the "cemetery" A 
(not included in X), and stays there forever. (Here £ takes non-negative real 
values and r\ takes non-negative integer values). The value of £ (and hence 
rj) can be infinite for certain sample paths, in which case the path does not 
terminate. Note that the difference 1 — ^2 ye x P%y represents the probability 
p(x, A) of a jump from x to the "cemetery" A for the embedded Markov 
chain {Y m } m > . Also, conditional on {Y m } m > , the intermediate jump times 
{S^i — Si}i>Q are independent and have Ex'ponentiali—qy^ distribution 
for i = 0,1,2, 

Let us prove by the method of contradiction that under ([T|) or (T5]), J2^=o P n < 
oo and hence Q~ l exists. For this it is enough to show that all states are 
transient. Suppose x G X is recurrent. The assumptions ([T]) or §2§ imply 
that 3n such that P n (x, A) > 0. This implies by [3j Theorem 3.4] that start- 
ing from the "cemetery" A there is a positive probability of reaching the 
state x, which is a contradiction as the "cemetery" A is absorbing. Hence, 
o P n <oo and Q- 1 = (£" =0 P")(Q d ^)- 1 exists. 

The lemma below relates Q^ 1 to the expected infinite occupation times 
for the process {X t }t>o- 

Lemma 2.1 

-Q- 1 (x,y)=E x [iy oo ], Vx, y E X. 
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Proof Firstly, Q = Q dia 9(I - P) leads to 

Q-i = (j^pA K/"" 1 



v n=0 



By decomposing the path of the Markov chain in terms of jump times, we 
get, 



E, 



/*oo 

/ l{X s =y}ds 

Jo 



= E, 



E, 



— ~E>x 



E, 



^2(S i+1 - Si)l {XSi =y} 

i=0 

oo 

^(Si+l - Si)l{ Yi =y} 

i=0 

oo 

E x [(/Sj+i — Si) | {^n}m>o] l{li=J/} 



i=0 

oo 



i=0 ^ 



The previous equality follows from the fact that conditional on {Y m } m > , the 
intermediate jump times {S i+ i— Si}i> are independent and have Ex , ponential{—qY i Y^) 

distribution for i = 0,1,2, 

This gives 



l {*s=y} 



ds 



— — Ej 

Qyy 



E 1 

,i=0 



{Yi=y} 



oo 

^yy ,-=n 



oo 

Note that P x denotes the probability distribution starting at the state x, 
while P is the transition matrix for the embedded Markov chain {l / m } m >o- 
Hence, 

" poo 
I l{X s =y}ds 

Jo 

= -(Q dia9 (I-P))-\x,y) 
= -Q- 1 (x,y)M 



E 
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3 History 



In a series of papers Dynkin [U El El H] proposed and built on his construction 
as a connection between random fields and Markov processes. Marcus and 
Rosen have used the refined knowledge about Gaussian fields (eg. continuity 
of sample paths) to develop fine properties of symmetric Markov processes 
(eg. continuity of local times) using Dynkin's construction. Their book 
[TT] gives a detailed and accessible account of their methods. Sheppard [13] 
uses Dynkin's isomorphism to give a proof of the Ray-Knight theorem on 
the Markovianity of one-dimensional diffusions. The properties of Markov 
processes can be utilized for analyzing the corresponding Gaussian fields. 
Ylvisaker [TT] uses Gaussian fields (amenable to Dynkin's isomorphism) as 
Bayesian priors for prediction and design problems, and makes use of the 
formulas relating the prediction properties of the Gaussian field to the cor- 
responding Markov process. Bolthausen [1] uses Dynkin's isomorphism as a 
tool in analyzing the limiting behaviour of the Gaussian free field. Eisenbaum 
[n] and also Marcus and Rosen [TT] have established variants of Dynkin's 
isomrphism. In the case of diffusions, Eisenbaum [8] shows that Dynkin's 
isomorphism theorem and the Ray-Knight theorems can be derived from 
each other. In [10] the authors use an unconditional version of Dynkin's iso- 
morphism to obtain a Ray-Knight theorem for a class of symmetric Markov 
processes. Diaconis and Evans [2] proposed a different construction by look- 
ing at — Q as the variance-covariance matrix instead of —Q^ 1 . Their con- 
struction yields Gaussian fields with negative individual covariances. 



4 Generalization of Dynkin's Isomorphism 

We again consider a Markov process {X t }t>o with a countable state space X 
and with a generator matrix Q as in Section 2. We introduce a "sign-matrix" 

5 such that 

S(x,x) = 1, S(x,y) = S(y,x), S(x,y) e {-1, 1} Vx ^ y e X. 

As explained in the introduction, if S(x, y) = 1, the transition from x to 
y is called a positive transition. If S(x, y) = — 1, the transition from x to y 
is called a negative transition. The "sign-process" H = {H t } t > is defined 
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by 

oo 

H t = Y[S(Y^ 1 ,Y i )l {Sl < t} , (with H = 1), 

i=l 

and H t = 1 if the number of negative transitions of X upto time t is even and 
H t = —1 if the number of negative transitions of X upto time t is odd. Also, 
the transition to the "cemetery" A from any state is a positive transition by 
default. 

Example If S(x, y) = 1 Vz ^ y 6 X, then H t = 1 Vt > and E = -Q" 1 . 
Example If <S(x,y) = -1 Vx ^ y G X, then # 4 = (-l)l{*> 1: Vt > 
and E = — (I + P)- 1 ^" 9 )- 1 (v From ©). For this particular case, the 
hyper-process {H t } t > is 1 between [S 2 i, S 2 i+i) and -1 between [52i+i> 
for i > 0. 

Define 

If := / l {Xs=XiHs=1 }(is - / l {Xs=X:Hs= _ 1} ds = / l {Xs =i}H s ds, t > 0, iG ^. 
Jo Jo Jo 

We interpret If as the net occupation time in the state x till time t (with 
J l{ Xs=2 . Hs=1 }(is and J l{x s=a ;,// 3 =-i}(is interpreted as the negative and pos- 
itive occupation times respectively). 
Define the matrix 

E 5 := (-Q0S)- 1 . 

Here o denotes Hadamard product i.e. elementwise product of the two ma- 
trices. Note that |Po«S| = P and X^o-^" 1 < 00. Hence (— Q o exists. 
Also, since — Q o S is a diagonally dominant matrix with positive diagonal 
entries, hence E 5 is positive definite. Note that, an infinite matrix is de- 
fined to be positive definite if all its finite principal submatrices are positive 
definite. 



Lemma 4.1 



Z s (x,y) = E x 



oc 



l{x s=y }H s ds 







Vx,yeX. 



i.e. Y> s (x,y) is the expected net occupation time at y starting at x. 
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Proof Firstly, 



This gives 



QoS = {Q dia9 {I - P))oS 
= Q dia9 (I-PoS) 



S 5 = (-Q o S)- 1 = (PoS- t)-\Q dia9 )-\ 



By decomposing the path of the Markov chain in terms of jump times, we 
get, 



E, 



i{x s = y }n s 



Hcds 



E, 



= E, 



= E, 



- Si)l { x Si = y }H Si 

i=0 

oo 

^(^i+l - >S'i) 1 {Y l =S/}#S l 
i=0 

oo 

^Ej, [(S i+ i — Si) | {F m } m >o] l{Y i=y }H s . 



The previous equality follows from the fact that 

i 

Hs^HSiYj^Yj) 

3=1 

is a function of {Y m } m > . This gives, 



E, 



-{x„=y}-n s 



E 3 



1 {y i =y} fi 's i 



-E, 



i=0 



j oo 

«ro i=0 

The exchange of sum and expectation is justified by the fact that |if s 
and ZZo [l{y i= y } ] = (/ - P)- 1 ^, y) < oo. 



= 1 
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Let us calculate E x \l^ Yl =y}Hs^\- 



im=v>II 5 ft-i' y i) 

n^w-iw n^- 1 '^ where y ° =x iVi= y- 

W,V2,—,Vi-l€X j=l j=l 
i 

YlPvi-ivj s (vj-i>yj) where vo = x ^Vi = y- 

= (Po5) l (x,y) 

Hence, 

/■oc 

E x [/1] = E x [/ l {Xs=2/} # s ds] 
Jo 

oo 

= -— X>oS)<(x,y) 
g ™ i=0 

= -(g^CJ-PoS))- 1 ^^) 

= S 5 (x )2/ ) 

The proof is complete. ■ 

We next prove the isomorphism theorem (jlj) for a zero mean Gaussian process 
{Z^}xex with variance-covriance matrix £ = (— QoS)^ 1 and an independent 
realization {X t } t >o of the Markov process with generator Q. 



E x [l{ Yl =y}H Sl ] - E x 



4.1 The finite case 

We consider the case when X is finite. We proceed similarily as Dynkin [4] 
and first consider functions of the form 



where d = {d u } u& x is arbitrary with d u > 0, Vw G X. Let £) denote the 
diagonal matrix with diagonal entries {d u } u( zx- Then, 



E 



2* V 



2 ^ 



i*i (v / 2^) l ^ | rfet(-Qo5) 
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det(D -QoS) 



det(- 


QoS) J 


det((D 


-Q)oS) 


det(- 


-QoS) 


det((D 


-Q)oS) 



Z X Zy 



z_ L (D-QoS)z_ 



Z X Zy 



2 dz 



z 1 {(D-Q)°S)z 



dz 



det(-QoS) 



((D-Q)oS)- 1 (x,y) 



Note that E 
Hence, 



det((D-Q)oS) 
det(-QoS) 



by a similiar calculation as above. 



E 



E 



— V rl £a 

g l^uex au 2 



((D-Q)oSr\x,y) (8) 



Note that —(D — Q) is the generator of a Markov process {X t } t > with the 
same structure as {X t }t>o except that at every state x G X, there is an 
additional killing rate of d x . Let {Y m } m > be the embedded discrete-time 
Markov chain and {H t } t > the hyper-process corresponding to ({X t } t > ,S). 
Let us establish the change of measure formula from {F m } m > to {Y m } m > Q . 



P x {Yi = yi,Y 2 = y 2 ,...,Y n = y n } = [J — 



n 



I Ivi-iVi-i + ^Vi-l 



(with = x) 



Px{Y\ =yi,Y 2 = y 2 , ~.,Y n = y n } 



Hence, 
E X [F(Y 1 ,Y,,...,Y„)]=E. 



n 



-_l %i-iyi-i + 

for each bounded Borel measurable function F. 

As in the introduction, let us define the measure fi xy by 

Hx V {C} = -Q^(x, y)P x {C | rj < oo, Y v = y} 



F(Y 1 , Y 2 , Y n ) 



(9) 



(10) 



It is the appropriately scaled conditional probability measure given that the 
process X enters the "cemetery" A eventually with y being the last state it 
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stays in before being killed. Note that, 

VxyiYi = y u Y 2 = y 2: Y 3 = y 3 , Y n = y n ,<q = n} 

P x {Yi =Vi,Y 2 = 1/2, Y 3 = y 3 , Y n = y n , Y n+l = A}l {yn=y} 



-Q-\x,y)- 



P x {r] <oo,Y ri = y} 



_ n -i ( ^ p A Y i = yi,Y 2 = y 2 , ...,Y n = y n }l {yn=y} p(y,A) 

v [x,y) Y,n= PAY n = y,Y n+1 = A} 

-Q~ 1 (x,y)p(y,A) 



P x {Yi = yi, Y 2 = y 2 , Y n = y n }l {yn=y} 



ZZoP*{Y n = y}p(y,A) 

= — Px{Yi =y u Y 2 = y 2 , Y n = y n }l {yn=y} 

lyy 

The previous equality follows from the fact that — Q^(x, y) = J2^Lo Px{Y r 

y}. 

Hence, 

f F(Yi, Y 2 , Y v )l {v=n} dfi xy = —E x [F(Y 1} Y 2 , Y n )l {Yn=y} ] (11) 

J Qyy 

for each bounded Borel measurable function F. 

We combine these results to evaluate ((D— Q)oS)~ 1 (x, y) in terms of {1%q} U £X 
and Hoo. Let \i xy be the measure defined in (fTOj) . 

Lemma 4.2 

((D-Q)oS)-\x,y) = J e-^^H^y. 

Proof Firstly, we observe that under fi xy , r\ < oo and hence Hoq = Hs v , 
which is a measurable function of Y 1 ,Y 2 , ...,Y V . Also, 

u&X i=0 

Hence, 
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oo „ 

/ e-^o^ + r-s,) Hsnl{v=n}dfixy 

A J 



n=0 

oo 



J2—V x e-^^(s l+1 -s l)Hs j {Yn=y} (v prom jm)) 

n =o q yy 



Qyy 1 L 



m>0 



n=0 

OO 



E— 

n=0 ^ 



n 

,i=0 



-QYiYi 



-q Yi Yi + d Yl 



H Sn l{Y n =y} 



The previous equality follows from the fact that conditioned on {Y m } m > , the 
intermediate jump times {S i+ i—Si}i> are independent and have Ex n ponential{—qY i Y^ / 
distribution for % = 0, 1, 2, 

Hence, with {Si}i>o denoting the random transition times for {X t }t>o, we 
get, 



oo 

= E 

n=0 


1 


~Qyy + ^ 


oo 

= E 

n=0 


1 


— Qyy + ^3/ 


= E, 


oo -, 

n=o ~ q yy 


= E x 


" POO 

Jo 


= ((D 


-Q)oS) 



E, 



n 



-QY^xYi-x 



The previous equality follows from the fact that {X t } t > is a Markov process 
with generator —(D — Q) which satisfies ([T]) or <$fy. 
Hence proved. ■ 



It follows from this claim and ([8]) that 



E 



E 
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Hence, 



E 



(zg) 2 



E 



" 2-duEX du 



(zg) 2 



■ m 



The set of functions Fjw) = e^^^ xduWu , where d = {d u } u€ x is arbitrary 
with d u > 0, Vit G A", generate the Borel a-algebra in M) x \ and they form a 
closed class under multiplication. Also, the set of functions F for which 



E 



( {Z s u f 



u G X 



E 



df-lxy i 



is a linear space closed under bounded convergence and under monotone 
convergence. Hence, for each bounded Borel measurable function F : M} x \ — > 



E 



Z t Z y F y 



( ( z uY 



u G X 



E 



dfJ'xyi 



(12) 

where {Z x } x& x is a zero mean Gaussian field with variance-covariance matrix 
given by (— Q o <S) _1 , and {/^Jug,*' are the occupation times of a realization 
{X t } t > independent of {Z x } x( z X of a Markov process with generator Q. Also, 
in this case the map (Q,S) — > (-Q o S)^ 1 is one-to-one, because 

(-QioSi)- 1 = (-Q 2 oS 2 )- 1 <s> Q l oS l = Q 2 oS 2 

Qi = Q2, <Si = $2 

The previous statement is justified by the fact that Q i: i = 1, 2 have negative 
off-diagonal entries and Si, i = 1, 2 have entries that equal 1 or —1. 

Let us now turn our attention to the problem of predicting the above Gaus- 
sian field given observations in a proper subset A C X, and proving the 
identities ([5]) and ([6]). We do not require the assumption of independence of 
{X t } t >Q and {Z x } x( zx for these calculations. Let B := X \ A. Note that, 



7 s , z s, 



Since S 5 



^[ Z B I ^A 

-Q o it follows that 

---{(-QoS)bb}- 1 



BA AA 



-QoS) 



BA- 
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By slightly detailed, but straightforward matrix computations as in Lemma 
I4TTI it follows that 



{(-QoS) BB }- L (b, b') = E 



OO 



i=0 



—Qb'b' 



where R A is the first time (greater than equal to Si) that the Markov chain 
{X t } t > hits A and H Sl is the "sign-process" evaluated at the i th jump time 
Si, for i > 0. Hence, 



^BA^AA <A a ) - ^2 E b 



b'eB 

OO 



qv, 



-Qb'b' 



5(6', a) 



1 {S t <R A } 1 {Y l =b'}H Sl 

i=0 

= ^ ^2 Efc [ 1 {Si<RA}' i -{Yi=b'}'i-{Y i+1 =a}Hs i+1 ] 

The previous equality follows by conditioning, the Markov property and 

n<(v ; 

Hence, 



P b ,{Y 1 = a} = ^=p(b\a). 



^ba^aa 1 (b,a) = E b 



Ei, 



OO 

^ ^-{Si<RA,Yi£B,Y i+1 =a}Hs i+i 
i=0 
oo 

^ 1 {i?A=5 , i+ i,y i+ i=a}^5 1+1 
,i=0 



It follows that, 

Z BA Z s AA ~\b,a) = {(-QoS) BB }- 1 (-QoS) BA = E b 
Hence, V6 e B, 



1 {X RA =a}H RA 



E[Z*\Zf, aeA] = J2v b [l 

aeA 



{X Ra =o}H Ra 



Zi 



■ (13) 



(14) 



Also, 



Var\Z% | Zf 



^BB ^BA^AA ^AB 

{(-QoS)^}- 1 
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Hence, 

Cov(Zf, Z§ \Z s a , aeA) = E b 

It follows that, 

Cov(Z s h ,Z s h ,\Z s a , aeA) = E, 

4.2 The infinite case 



,i=0 



^{S l <R A \ l {Y l =b'}H Sl 
—Qb'b' 



V6, b' G B. 



' POO 

/ l{Xs=b'}H S l{s<R A }ds 

Jo 



, V6, 6' G B. 

(15) 



We now deal with the case when X is countably infinite. To prove (jlj) in this 
case, arbitrarily fix a finite subset Xf C X. Note that the variance-covariance 
matrix for {Z^}xex f is given by 



£/ := (-(<3 ° S)*^ + «S) Ar/A -c{(g o S)#pi-c} o 



1-1 



Hence for x,y G Xf, it follows by a similiar calculation leading to (JSJ) that 
for arbitrary gJ w > 0, u E Xf, 



E 



E 



Here Df is a diagonal matrix of dimension | Ay | with diagonal entries d u , u G 
Af/. We now prove a claim which will help us prove that (Df + EjW,?/) 
is indeed (D — Q o (S) _1 (a;, y), where D is a diagonal matrix of dimension \X\ 
with diagonal entry d u if u G Ay and otherwise. 

Claim 4.1 Lei A C X be finite and 

(Q o S) A := (Q o S)aa - (Q o o 5)bb} _1 (<9 o «S)ba. 

If Q satisfies OP or ([M), 



(Q 0l S) A (a,a') 



g aa (l - E a [l {x ^i?^]) t/a 



-gaaE a [l { x fl , =a'}#R 4 ] 



if a ^ a' 



and, 

(b) {(Q o S) A }-\a, a') = (Qo 5) _1 (a ; a) Va, a' G A 



15 



Proof Throughout the proof, the absolute convergence for various infinite 
sums will be taken care of by the fact that P n < 00 (because Q satisifes 

(CQ) or (J2J)). Note that the prediction formulas derived for the finite case in 
Section 4.1 go through for the infinite well, if A is finite. Recall that 

Ra is the first time the process {X t } t >o hits A after the initial state. Hence 
from (fT51) and strong Markov property, 

-(Q oS) A (a,a') = -q aa < -^2q ab S(a,b)E b [l { x RA =a>}H RA } 

beB 

= -Qaa' + qaa^a[^{X R , A =a'}^-{R A >Si}HR A } 

-q aa (l - E a [l { x RA=a }H RA }) if a = a' 

qaaE> a [l{x R , A =a'}HR A ] if a ^ a! 

This completes the proof of (a). This also gives (Q o S) A = Q d AA 9 {I — P A ) 
where P A (a,a') := E a [l{x fl =a'}H RA ], Va, a' G A, and Q dtag is the diagonal 
matrix with the same diagonal entries as Q. Let R n A denote the time of n th 
return to A. It follows that, 



00 

{(Qo«S)VW) = — y2(P A ) n (a,a' 



qaa n=0 
j 00 n—1 

^ aa n=0 a , ai ,...,a n £A i=0 
a,Q = a ,afi=a f 

OO 



— ^E a [l {Xii „=a'}^i 



Qaa ,-, 

n=0 



The previous equality follows by the definition of {H t } t > and repeated ap- 
plication of the strong Markov property. Observing that Xs i = a' only if 
Si = R\ for some n > 1, we get that, 

-. 00 

{(Qo l S) A }- 1 (a,a') = — J2V a [l { x Si=a > } H Si ] 



qaa . =Q 



-E„ 



" poo 
/ l{x 3 =a'}# S ds 



= (QoS)-\a,a') 

The previous equality follows from Lemma 14. 1[ The proof of (b) is now 
complete. ■ 
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Note that —(D — Q) is a generator matrix that satisfies (pQ) or <^ (because 
Q satisfies one of these conditions). Applying Claim I4TT1 for —{D — Q) with 
A = Xf, we get that, 

(D f + T.f)- X {x,y) = {(D - Q) o S)-\x,y). 
By imitating the proof of Lemma [4.21 we get 

{{D-Q)oS)-\x,y)- I »-Zue Xf ^ 
Combining everything, 



Hoodfj, 



E 



v 



E 



Note that the set of functions F^(w) = e~^^ xduWu , where d = {d u } u( zx is 
arbitrary with d u > for finitely many u G X and d u = otherwise, generate 
the Borel cr-algebra in M} x \ and they form a closed class under multiplication. 
Also, the set of functions F for which 



E 



z s z s F u e x 



E 



d/J'xy j 



is a linear space closed under bounded convergence and under monotone 
convergence. Hence for each bounded Borel measurable function F : R'*' — ► 



E 



zf z s y F ( „ e ;v 



E 



4.3 Conditional Independence Property 

There is another interesting property of Dynkin's isomorphism which is pre- 
served after introducing a "sign" matrix S. Let Q be the generator of a 
continuous time Markov process {X t } t >o, with a countable state space X. 
Assume Q' 1 exists and Q is symmetric. Let {Z^} xeX be a zero mean Gaus- 
sian field with variance-covariance matrix S 5 = (-Q o <S) _1 . 

Lemma 4.3 Lei A, 5, C fre disjoint subsets of the state space X , such that 
to go from any state in A to any state in C, the Markov process {X t }t>o 
has to pass through B. Then conditioned on Z% := {Z^} beB , the Gaussian 
random vectors Z S A := {Zf } a eA an d Zq := {Zf} c&c are independent. 
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Proof Fix a G A and c G C arbitrarily. By (1151) , 



/ l{x 3 =c}l{s<R B }H s ds . 

Jo 

Since the Markov process {X t } t > has to pass through the set B to go from 
the state a to the state c, 



Cov{Z s a ,Z s c \Zt beB) = E, 



l{x s =c}^{s<R B } = under P a . 

Hence, 

Cov{Z s a , Z s c | Zg, beB)=0. 

Since a G A and c G C were arbitrarily fixed, it follows that Z\ and 
are uncorrelated given Z§ . Two random vectors having a joint Gaussian 
distribution are independent iff they are uncorrelated. Hence, Z\ and Zf, 
are independent given Zg. ■ 

Bolthausen [1] uses this property in his analysis of the Gaussian free field. 



4.4 An Example: Ornstein-Uhlenbeck Process on N 

Consider the Ornstein-Uhlenbeck process {Zj} ieN defined by 

Z\ — ei, Zi = aZ^i + Si Vz > 2, where {£j}i>i are i.i.d. N(0, 1). 
Let S denote the variance-covariance matrix of {Zi}^. Then, 

After some manipulations, we can establish that £ = — Q 1 where 

f-(l + a 2 ) iffc = Z, 
Q(M) = <a iffc = Z±l, (17) 

l> otherwise. 

If a > 0, then Q is the generator of a birth and death process. Hence, the 
Ornstein-Uhlenbeck process is connected to the birth and death process with 
generator Q by Dynkin's isomorphism. 
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Suppose we introduce a "sign" matrix S (as described earlier in this section) 
and for a > 0, look at a Gaussian field {Z 4 '} ieN denned by 



Z[ = e 1 , Z- = S(i-l,i)aZ' i _ 1 + e i , Vi > 2, where {^j^i are i.i.d. iV(0,l). 

It follows after some manipulations that the variance-covariance matrix of 
{Z[}i^i is given by (— Q o S)^ 1 , where Q is as specified in ( FIT]) . 
If S(k,l) = -1 Vk ^ I, then 

Z[ = s u Z[ = -aZ'^ + Si, Vi > 2. 

Hence, if a > 0, then the Ornstein-Uhlenbeck process on N with parameter 
—a is associated to a birth and death process with generator Q in ffTTj) . by 
Dynkin's isomorphism with "sign" matrix S such that S(k, I) = —1 \/k ^ I. 



An Algorithm for Computing the Predic- 
tion Coefficients 



^-{X RA =a}H RA 



We present an algorithm for computing the prediction coefficients E& 

in (THI) for calculating E[Zf | Z^ , a G A}. This algorithm can be described 
in two ways: 

• Graph theoretic description. 

Construct a graph on the vertex set X by putting an edge of weight 
Q(x,y)S(x,y) between vertices x and y, Wx ^ y G X. If any of these 
weights are 0, that by default means no edge is put between the cor- 
responding vertices. Put a loop of weight — Q(x,x) at each vertex 
x G X. Since we want to predict Zf given {Zf } a ^A we now proceed 
to remove all vertices not in {b} U A from this graph in a sequential 
fashion. Choose any vertex, say z not in {b} U A. If we remove z, i.e. 
if we behave as if z does not exist in the graph, this leads to forming 
a new edge between every pair x and y such that x and z, as well as 
y and z share an edge. The weight of this new edge is the product of 
the weights of these two edges divided by the weight of the loop at z. 
If there is already an edge between x and y, add the weight of this new 
edge to the existing one and combine them into one edge. Perform this 
procedure with all x and y sharing an edge with z (including the case 
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x = y). So we get a new graph with vertex set X \ {z} and edge set 
as described above. Note that — Q oS is a diagonally dominant matrix 
with positive diagonal entries and hence for the old graph, the weight 
of the loop at any vertex dominates the sum of the absolute weight val- 
ues of the the edges emanating from that vertex. As we will see later, 
the new grpah has the same property. We continue choosing vertices 
and removing them by using the above procedure until we are left with 
the vertex set A U {b}. The coefficient of Zf in E[Zf | Zf , a G A] is 
precisely the weight of the edge joining a and b divided by the weight 
of the loop at b, for every a in A. 

• Analytic description. 

We can describe the above algorithm analytically as follows: 

1. Start with V — X, M = -Q o S. 

2. Choose z eV\{AU{b}}. 

3. M(x, y) = M(x, y) - M(x ^ z) Vx, y G V. 

4. Remove the z th row and the z th column of M. 

5. V — > V \ {z}. If V 7^ A U {b} goto step 2, otherwise stop. The 
coefficient of Zf in E[Zf \Zf, a E A] is for every a G A. 

The above description tells us that our algorithm is essentially the sequential 
process of evaluating the Schur complement (— QoS)vv ~ (~ Q°S)vv c {(— Q° 
S)yc V c}~ 1 (—Q o S)yc V (and ending at V = A U {b}) by reducing rows and 
columns. Since the Schur complement of a diagonally dominant matrix is 
also diagonally dominant, the matrix M is a diagonally dominant matrix at 
every step of the algorithm. The proof of this algorithm can be obtained 
immediately by observing two facts. Firstly, = (-Q o S)^ 1 implies that 

(S^)" 1 = {-QoS)vv-{-QoS)yv,{(-QoS)vcyc}- 1 (-QoS)ycy for every V C 

Hence, when we stop the algorithm, the matrix M is the same as (Sy- y ) _1 
with V = A U {b}. Secondly, if Y ~ MVN n (0, T), then 

m\Y J ,j^ i ]=J2=f^Y J . (18) 
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Since {Z$} veV is MVN\ V \(0, ^v V ) and M = ( s vv) _1 , & follows by (HHD that 



Hence this algorithm is not all that mysterious. If \X\ = n, the worst case 
running time of this algorithm is 0(n 3 ). One nice property of this algorithm 
is that at any step of the algorithm with vertex set V and corresponding 
matrix M, 



Hence, if we want we can obtain the prediction coefficients given {Z^} v ev 
for every V C X that comes up in the course of this algorithm. 

5.1 An Example of Prediction with Independent Er- 
rors at the Observed Values 

Consider the Ornstein-Uhlenbeck process on N with a = 1, i.e. 

Z\ = £i, Zi = Zi-i + EiVi > 2, where {£i}i>i are i.i.d. N(0, 1). 

This process is same as the Gaussian free field on N. It follows that the 
variance-covariance matrix S of {Zi}i £ ^ is given by 



Suppose we observe the values of the process in the set V = {ni, n 2 , n^} 
(where n« < rij if % < j), but with an independent additive error S{ at 
the point i — l,2,...,k, where {ei}i<i<k are i.i.d. N(0, a 2 ). With these 
observations, we want to predict the process {Zj} ieN i.e. we want to compute 
the expectation 





E(k,l) = k Al V7c, / G N. 



E[Z n | Z ni + £i, Z n , 2 + £2 



Z nk +e k ), Wn e N. 



It is known that 



E[Z n I Z ni + £j, z 



1,2, ...,k] 



Sriy(Syy + O" J|v|) ^y- 
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Since S(rij, n) = A n Vz = 1, 2, /c, we would like to compute a simplified 
expression for (Syy + °" 2 -f|V|) -1 - We utilize the structure of Syy for this 
purpose. 

Y, vv = UDU T , 

where, U(i,j) = l{j>j}, VI < i,j < k and D is a diagonal matrix with 
D(i, i) = rii — nj_x VI < i < A; (where no = 0). It follows that 

S V y + a 2 J|y| = UAU T , 

where A is the tridiagonal matrix with 

{rii — Ui_x + icr 2 if i — j, 
-a 2 if|«-j| = l, (19) 

otherwise. 

Note that, 

fl if * = J, 

U-\i,j) = l -1 if z = j + 1, (20) 

1 otherwise. 

Let r = ex 2 , n = (ni + cx 2 )cr 2 , r; = (n» - n^i + ia 2 )rj_i - rj_ 2 for z = 
2,3, ...,k. By the explicit formula for the inverse of a symmetric tridiagonal 
matrix in [T2] . 

= '-7 (21) 

I A if i > j. 

Hence we obtain 

k k 

E[Z n | Z ni +£j, i= 1, 2, k] = ^2(%2'Yij(nAn j ))(Z n . + E i ), 

i=l j=l 



where, 

(ri-r i _i)(r fc _ j -r fc _ i _i) 



if i < j, 



Jij = < r-»-i(rfc-i-2r fc _i_i)+rir fc _i_i ^ ^ _ ■ (22) 

Iji if i > j- 

As is clear from this example, introducing errors leads to non-trivial changes 
in the prediction coefficients. It is hard to find a general formula which 
expresses these changed coefficients in terms of the associated Markov chain. 
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